Section: New Results

Multilingual POS-tagging and Parsing

Participants : Éric Villemonte de La Clergerie, Djamé Seddah, Benoît Sagot, Héctor Martínez Alonso.

Our participation in 2017 to two international shared tasks (CONLL UD and EPE—the latter in collaboration with Stanford University) led us to develop a new generation of statistical multilingual NLP tools, in particular for POS-tagging and for Parsing [29]. In particular, the CoNLL shared task involved 80+ datasets covering 50+ languages (including low-resource and no-resource languages) and, for some languages, various genres.

For POS tagging, we have developed a new feature-based POS tagger, following our previous work on MElt [56], [72]. This new tagger, named alVWTagger, uses the Vowpal Wabbit system for training linear POS models, resulting in an important drop in training times. This has allowed us to better explore the feature set space based on development data for each and numerous ways to encode the information provided by external morphological lexicons, resulting in better tagging results. We also developed a derivative of this tagger for performing tokenisation and sentence segmentation. Experiments on the development sets of the CoNLL shared task allowed us to chose the best setting for each corpus between several configurations, by using the UDPipe baseline (provided by the shared task organisers) or alVWtagger for each of the 3 subtasks (tokenisation, segmentation in sentences, UPOS tagging). As a result, we ranked 3rd (out of 33 participants) in the UPOS tagging ranking of the CoNLL shared task, and 5th for the tokenisation subtask and 6th for the sentence segmentation substask. Moreover, later improvements in the parsing models resulted in alVWtagger being more often used than for the official run, with improved results (unofficial post-campaign ranking on UPOS tagging: 2nd/33).

In parallel, we have developed a neural POS tagger based on Barbara Plank's LSTM tagger, by exploring the impact of integrating lexical information extracted from morphological lexicons within the neural architecture. We showed that such information improves POS tagging on average [25]. A careful comparison of this neural tagger, alNNtagger, w.r.t. alVWtagger is yet to be carried out, but preliminary experiments tend to show that both taggers perform similarly on average. This is likely because POS tagging is a relatively easy task for which the manual design of adequate features is relatively easy. As a result, using a neural architecture, which has the advantage of learning the optimal features rather than relying on manually crafted ones, does not result in massive improvements as observed in many other NLP tasks and beyond.

For Parsing, DyALog-SR, a feature-based parser on top of DyALog system, was extended (into DyALog-SRNN) to integrate predictions proposed by deep neuronal layers, based on a global char LSTM and a word bi-LSTM. Based on the results of the CONLL UD shared task, further extensions were added to DyALog-SRNN, namely an adaptation of Stanford's winner system (based on a bi-affine prediction of word governors) and a version of the Maximum-Spanning Tree (MST) algorithm, allowing us to move from the 6th place (for parsing) to an unofficial post-campaign 4th place.

The new version DyALog-SRNN has preserved the functionality of DyALog-SR to produce (deep) dependency graphs rather than standard shallow dependency trees. This functionality was used during the EPE (Extrinsic Parsing Evaluation) shared task to test several dependency tree and graph representations for several downstream application tasks [28].

The goal of that collaboration with the Stanford NLP team was to evaluate the usability of several representations derived from English Universal Dependencies (UD), as well as the Stanford Dependencies (SD), Predicate Argument Structure (PAS), and DM representations. We further compared two parsing strategies: Directly parsing to graph-based dependency representations and a two-stage process of first parsing to surface syntax trees and then applying rule-based augmentations to obtain the final graphs. Our systems used advanced deep learning techniques on top of state-of-the-art preprocessing and part-of-speech tagging. Overall, our systems performed very well and our results were ranked first and third on that shared task (over more than 20 submitted systems). The main advantage of that shared task was to provide an extrinsic evaluation scenario which consisted in extracting relevant information for information retrieval from speech and biomedical data, as well as opinion mining.This showed the relevance of our approach and the interest of producing graph-based representations to downstream applications that were developed for tree-based structures.

In particular, it showed the interest of deeper syntactic representation instead of shallow ones. In parallel with these efforts, work was also carried out on the issues related to polylexical units in parsing [17]. Moreover, the International Journal of Lexicography has accepted a paper written in collaboration with three other European research centres on the interactions between NLP and lexicography on polylexical units (to appear in 2018).